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Executive Summary 
SPIRIT Program 
Year 2 


In 2002, the Missouri Department of Mental Health (DMH) initiated an ambitious project 
to implement evidence-based substance abuse programs in five selected Missouri school 
districts throughout the state. The project, the School-based Prevention Intervention and 
Resources IniTiative (SPIRIT), proposed to delay onset and decrease use of substances, 
improve overall school performance, and reduce incidents of violence. Outcomes were 
achieved through implementation of evidence-based prevention programs in kindergarten 
through 12" grades. These programs included PeaceBuilders, Positive Action, Life Skills 
Training, Second Step, and Reconnecting Youth. Findings after two years of project 
implementation show the following promising results: 


Over 4,000 children participated in the SPIRIT programs over the past two years, 
representing one of the largest in-school substance use prevention efforts in the state of 
Missouri. 


Teacher enthusiasm for the SPIRIT program was widespread. Teachers saw increases in 
positive behaviors and appreciated the additional support from the prevention providers. 
Comments from focus groups conducted at all programs this year included: 


“Kids say it makes them think about what’s right and what’s wrong.” 


“They take responsibility easier than they used to. Kids are doing stuff wrong, but they 
know it, they can admit it, and they are learning to solve it.” 


“[The prevention specialist] is a wonderful asset. She has been instrumental in helping 
teachers—she works with the kids to help them understand the importance of education. 
She is someone who cares.” 


“The kids are excited when [the prevention specialist] comes in. He always portrays the 
positive. He is a good male role model and mentor, especially for kids who don’t have 
one.” 


Between Fall 2002 to Spring 2004, kindergartners through 3™ graders showed slight 
reductions in reactive aggression (aggressive actions that result when an individual feels 
threatened). Typically, as children of this age group mature, aggression either increases or 
remains the same, but without an intervention, does not decrease. 


e Fewer children reported zero absences at all in Year Two compared to Year One.' 


Figure I. Number of Elementary School Absences, Year 1 & 2 
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e The number of disciplinary incidents among 4" and 5" graders was very low both years, 
and a slight reduction in incidents was detected, contrary to natural maturational trends. 


e When compared to a national sample of youth, middle school youth (n=164) participating 
in SPIRIT reported using significantly less substances after two years of prevention 
programming than the national sample. 





' Data for absences combines both unexcused and excused absences, therefore illness accounts for some of the absences 
both years. Unexcused absent data were not available across all schools. 


Figure II. Comparison of Substance Use for SPIRIT Youth with a National Sample 
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NOTE: National data are from the National Survey on Drug Use and Health, 2003. 


e Alcohol use rose slightly from Year One (Time 1 and Time 2) to Year Two (Time 3 and 
Time 4) for youth participating in SPIRIT. Use rates for the SPIRIT sample were 
considerably lower than national averages. 


Figure III. Percentage of High School Youth Reporting 30 Day Alcohol Use 
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e Grade point averages for both middle and high school students participating in SPIRIT 
increased slightly, though not significantly, from Year One to Year Two. 


Figure IV. Comparison of Middle and High School GPA, Year 1 & 2 
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e Middle school students showed decreases in the amount of bullying between Fall 2002 
and Spring 2004. They also showed reductions in the amount of pushing, shoving and 
hitting. These results are very encouraging given that students usually increase these 
behaviors in middle school. 


Figure V. Percentage of Middle School Youth Reporting Bullying Activity, 
Year 1&2 
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e High school students (n=171) showed even greater reductions in bullying behaviors 
between Fall 2002 and Spring 2004. 


Figure VI. Percentage of High School Youth Reporting Bullying Activity, 
Year 1&2 
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e The average number of disciplinary incidents in the high schools decreased slightly 
between Fall 2002 and Spring 2004, contrary to natural maturational trends. 


Figure VII. Average Number of Disciplinary Incidents for High School Youth 
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SUMMARY AND RECOMMENDATIONS 


Preliminary results of SPIRIT are encouraging. The project involved a large number of 
students from Kindergarten through 12" grade taking part in evidence-based substance use 
prevention programming over the past two years. The programs were successful in 
reducing the rates at which youth typically increase their substance use, reducing rates of 
bullying and physical violence, and slightly reducing disciplinary incidents in the sample 
of youth who completed all four survey instruments. Though not significant, GPAs and 
disciplinary incidents both improved as well, suggesting a positive trend for SPIRIT 
students. Teachers were enthusiastic about the programs and felt that they were making a 
difference in the lives of the children in their schools. 


Given that negative behaviors would typically increase during the transition periods as 
students move from elementary school to middle and then high school, these results 
suggest the programs have had an effect, over time, in reducing negative behaviors in 
school. Data being collected during the current academic year will add to the growing 
knowledge regarding the effectiveness of these evidence-based programs for Missouri’s 
youth. Recommendations to more fully and smoothly implement SPIRIT in the future 
include the following: 


Teacher buy-in, training and preparation. Teachers should be informed about new 
programming well in advance of program implementation and be trained thoroughly in the 
program. Incorporating teachers into the planning phase allows them to feel ownership in 
the program and creates less resistance to full implementation. Furthermore, they become 
better trained in prevention topics that may not be part of their normal professional 
development. 


Consistency in implementation. Studies have shown that consistent implementation of 
programming across staff leads to better program outcomes (Springer et al., forthcoming). 
Model programs have been tested, in most cases, with several different populations of 
youth, and therefore modifications needed for curricula are usually minimal. Adhering to 
the planned curricula, lesson plans, number of hours per week, and other features of the 
program design are preferred except in situations where the curriculum is clearly not 
applicable for the children being served. Future implementation of SPIRIT needs to stress 
the importance of consistency to all teachers and/or providers working with participating 
youth. 





Specific curriculum issues. The age and cultural appropriateness of selected curricula 
need to be addressed. Staff at at least one site perceived Positive Action’s curriculum was 
culturally inappropriate and at three sites was inappropriate for high school youth. 








Communication between teachers and providers. Increased communication between 
service providers and teachers is recommended to strengthen programs and provide for 
better program implementation. 


CHAPTER ONE 
INTRODUCTION 


Substance use is increasingly recognized as one of the nation’s most pervasive, costly, and 
challenging health and social problems. The use of alcohol and drugs has resulted in tens 
of thousands of deaths annually, with a cost in terms of lost earnings alone estimated at 
over $200 billion dollars annually.' Additionally, the use, and particularly the early use, of 
tobacco, alcohol, marijuana, and other illicit drugs is intricately entwined with serious 
personal and social problems, including school failure, crime, family violence and abuse, 
and a host of other such problems constituting a continuing national tragedy. 


Over the past 30 years, prevention researchers have made significant strides in better 
understanding the causes of adolescent substance use. Aggression, lack of self-control, 
low social competence, low school and family bonding, poor parental supervision, poor 
social skills, lack of adult support, low academic achievement, and a host of other factors 
have been shown to be related to adolescent substance use. In turn, model programming 
that bolsters individual, family, school and community factors shown to help adolescents 
resist substance use have been developed (see www.samhsa.gov, http://ojjdp.ncjrs.org, 
and www.ed.gov) and are being disseminated nationwide to move the prevention field 
toward more evidence-based prevention practice. 





In 2002, the Missouri Department of Mental Health (DMH) initiated an ambitious project 
to implement evidence-based substance abuse programs in five selected Missouri school 
districts. The program, School-based Prevention Intervention and Resources IniTiative 
(SPIRIT), proposes to delay onset and decrease use of substances, improve overall school 
performance, and reduce incidents of violence. SPIRIT engages schools, substance abuse 
providers and the DMH in prevention activities. Outcomes are achieved through 
implementation of evidence-based prevention programs in kindergarten through 12" 
grades. The Missouri Institute of Mental Health (MIMH) is conducting the project 
evaluation. 


The five school districts participating in SPIRIT are: Hickman Mills C-1, Jennings, Knox 
Co. R-I, New Madrid Co. R-I, and Carthage R-IX. One provider agency per district was 
selected, through competitive bid, to partner with the schools. Providers are responsible 
for providing or supporting implementation of the intervention, data collection, and to be 
available for screening of students and referral to appropriate treatment or intervention. 


Substance abuse prevention interventions were chosen from a menu of model programs. 
For the first year of the project, four prevention programs were chosen jointly between the 
schools and the providers: PeaceBuilders, Positive Action, Life Skills Training, and 
Reconnecting Youth.” To implement the programs, school districts could choose to have 





* More information regarding these prevention programs can be obtained at the following websites: 
www.peacbuilders.com; www.positiveaction.net; www.lifeskillstraining.com and 
www.son.washington.edu/departments/pch/ry/curriculum.asp. 
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either providers or teachers deliver the curricula. Some program changes were made for 
the second year of implementation. In the Jennings School District, Second Step, a model 
violence prevention program (see www.cfchildren.org/ssf) was selected to take the place 
of Positive Action in the kindergarten through 6" grades. Reconnecting Youth had been 
dropped at the alternative high school after one semester of implementation at this site as 
well. Knox County alternative school added Reconnecting Youth, implemented solely 
through selected lessons delivered one time per week. In the New Madrid district grade 6 
was added to the program. 


Although the methods, components, targeted behaviors, and comprehensiveness of the 
programs differ, the goals of all of the selected model intervention programs are the same. 
Their objectives are to prevent, delay, and reduce substance use, prevent early onset of 
substance use, increase school attendance, and reduce incidents of violence. Two of the 
selected programs, PeaceBuilders and Positive Action, focus on the entire school 
community and, in addition to affecting change in the individual student, seek to change 
the climate in the school. Life Skills Training, Reconnecting Youth and Second Step are 
oriented toward improving the behaviors of youth, with Reconnecting Youth, an 
indicated/selective program, specifically targeting youth with academic and/or behavioral 
issues. Since only the individual and school components of multi-systemic programs were 
implemented during the first two years, evaluation instruments established a common 
baseline to measure program impact on an individual level, within and across the school 
districts, and on the common defined goals. 


Enrollment in the SPIRIT Program 


Over the past two years, a substantial number of students across the five school districts 
have participated in SPIRIT. As shown in the chart below, over 4,000 children and youth 
have participated in the program to date, with the largest number of participants coming 
from urban sites [Jennings (St. Louis area) and Hickman Mills (Kansas City area)]. 


The three subsequent charts present the number and percentage of children participating in 
the SPIRIT evaluation. There are 2,250 children and youth participating in the evaluation, 
with the majority coming from elementary and middle school. Only one district has 
programs for all high school students. One district does not have a high school component, 
two districts have programs only in selected high school grades, and one district is 
implementing Reconnecting Youth for a limited number of students. Participation in the 
evaluation was contingent upon obtaining consent from parents and assent from children 
and youth. 
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Figure | shows the total enrollment of students in the SPIRIT project by district 


Figure 1. Total Enrollment in SPIRIT by District, All Grades 
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Of the total, 2,250 have participated in the evaluation during the course of the study (Figure 2). 


Figure 2. Total Number of Evaluation Participants by Grade Level 
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Figure 3 displays the percentage of students in each school district who are participating in 
the evaluation. 


Figure 3. Percentage of Students in SPIRIT Evaluation by District (n=2250) 
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Evaluation Methods 


The SPIRIT evaluation, conducted by the Missouri Institute of Mental Health’s Division 
of Child & Family Mental Health Services Research, is a multi-method, longitudinal 
evaluation that includes the collection of qualitative and quantitative data. Data sources 
include 1) self-reported youth behaviors, including substance use and other related 
behaviors; 2) focus group interviews with teachers, providers, and principals; 3) data 
documenting program fidelity and amount of program contact; and 4) data from the 
schools documenting grades, absences, and disciplinary incidents. Youth were surveyed 
in the Spring and Fall of the 2002-2003, and 2003-2004 academic years. Focus groups 
were conducted in the Spring of 2004. Fidelity and contact information has been collected 
regularly since program start-up. 


1 


Organization of the Report 


DMH is advancing the field of prevention in Missouri by introducing several well- 
known, evidence-based programs into five selected school districts around the state. 
This report documents the findings from the second year of implementation of the 
SPIRIT program. The structure of the remainder of the report is as follows: 


Chapter Two: Program Description and Fidelity. Chapter Two describes the 
individual model programs, and presents data related to the amount of participant 
exposure to program services. 


Chapter Three: Site Visit and Focus Group Findings. This chapter presents 
information gathered from focus groups conducted during Year Two from teachers, 
providers, and staff at the participating schools. 


Chapter Four: Outcome Findings. This chapter presents descriptive data regarding 
the sample and over-time findings from individual youth, reporting changes in substance 
use, protective and risk factors, grades and disciplinary actions. 


Chapter Five: Summary. This chapter summarizes the outcome findings with 
recommendations for the upcoming year. 
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CHAPTER TWO 
PROGRAM DESCRIPTION AND FIDELITY 


A. PROGRAM DESCRIPTIONS 


The Missouri SPIRIT project is implementing five model prevention programs in five 
school districts throughout the state. This chapter describes these programs, including 
program goals, model program status information, target population, curriculum content, 
program intensity and duration. Information on the quality of implementation is also 
discussed. 


PeaceBuilders 


Description 

PeaceBuilders is an elementary school violence prevention program that seeks to change 
the school climate by reducing student aggressive behaviors, increasing pro-social 
behaviors, and increasing academic performance. PeaceBuilder lessons are loosely 
defined with regard to length or number of sessions, but there are many suggestions for 
curriculum integration and use of the principles in handling specific situations, 1.e. 
interpersonal problems, playground incidents. The program is built on the concept of 
infusing the five PeaceBuilder principles and strategies into the regular curriculum and 
into school-wide special events. 


Model Program Status 

PeaceBuilders is designated as a “Best Practice” by the U.S. Center for Disease Control 
and a “Promising Program” by the U.S. Department of Education and the U.S. 
Substance Abuse and Mental Health Services Administration (SAMHSA). Studies show 
that children who have received PeaceBuilders in other school districts had fewer 
playground fights, fewer referrals to the principal’s office and suspensions, and 
decreases in visits to the school nurse caused by fights. 


Implementation 

PeaceBuilders is being implemented at two SPIRIT sites. One site uses classroom 
teachers to implement the program; the other uses a provider prevention specialist. 
PeaceBuilder lessons were conducted one time per week for 15-30 minutes with daily 
reinforcement at one site, and one time per week for 20-30 minutes at the second site. 
Kindergarten and Grade | lessons are modified at each site so that they are age- 
appropriate. At one site, a single presenter develops and teaches lessons on a common 
principle to students at all grade levels providing a uniform focus throughout the school. 
At the second site, teachers independently select the most applicable principle to use in a 
lesson developed to address specific classroom or playground issues or to stress a theme. 
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Positive Action 


Description 

Positive Action is a school climate program for all students aimed at reducing risk 
factors and increasing school bonding, improving student performance, and positively 
affecting behaviors including substance use, violence, and disruption. It is described as 
both a substance abuse and violence prevention program. The basic philosophy of the 
program is that “you feel good about yourself when you do good (positive actions).” 


The Positive Action Program has separate curricula for elementary, middle, and high 
school levels, and contains components for the family, and community. Only the 
elementary level curriculum has been the subject of published research. The elementary 
and middle school levels of the program are designed to be implemented daily in 15 
minute segments, while the high school curriculum requires lengthier segments. 


Model Program Status 

The elementary level program of Positive Action is designated an “Effective Program” 
by the U.S. Department of Education, as a “Promising Program” by Safe and Drug Free 
Schools, and as a “Model Program” by the Substance Abuse and Mental Health Services 
Administration (SAMHSA.). Research shows that implementation of Positive Action 
results in improved academic achievement and self-concept, and reductions in violence 
and substance use, suspensions, and truancy. 


Implementation 

Positive Action was originally implemented in four school districts at different school 
levels. At one site, providers implemented the program; at the other three, classroom 
teachers provide instruction. One site implements the program in selected grades every 
other week for 15-30 minutes; the others implement the program weekly ranging from 
10 to 60 minutes per week. One site opted to change from Positive Action to Second 
Step during the second year of implementation. At a second site, most middle school 
teachers opted out of Positive Action in the second year and chose a video program in its 
place. 


Second Step: A Violence Prevention Curriculum 


Description 

Second Step is a school-based social skills curriculum for students from pre-school 
through junior high. The goals of the program are to teach children to change attitudes 
and behaviors that contribute to violence by reducing impulsive and aggressive behavior 
and increasing social competence. There are 15-20 lessons for each grade level focused 
on empathy, impulse control, problem solving and anger/emotion management. It is 
recommended that lessons, based on class discussion and skill practice, be taught once 
or twice a week for 20-50 minutes depending upon students’ ages. A multicultural 
perspective is incorporated throughout the program. 
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Model Program Status 

Second Step has been designated as an “Exemplary Program” by the U.S. Dept. of 
Education, a “model program” by SAMHSA, and a Select Program by the Collaborative 
for Academic, Social, and Emotional Learning (CASEL). It received the highest rating 
of any elementary and middle school violence prevention programs in Drug Strategies 
evaluation, and recognition from the White House, New Jersey Dept. of Education and 
the Utah Office of Education. Research shows significant outcomes for preschool- 
kindergarten students in decreased verbal aggression, disruptive behavior and physical 
aggression, and in improved empathy skills and consequential thinking skills. It also 
shows decreased aggression on the playground and in conflict situations, decreased need 
for adult intervention, better anticipation of consequences, increased social competence 
and positive social behavior, and for girls, higher levels of empathic behavior in conflict 
situations. Less approval for physical, verbal and relational aggression and increased 
confidence in the ability to regulate emotions and problem solve were outcomes for 
middle and junior high school students. Second Step has demonstrated effectiveness 
with students of diverse racial and ethnic backgrounds. 


Implementation 

Second Step replaced Positive Action in the second year in grades K-6 in one district. It 
was believed that Positive Action did not deal with the primary prevention needs of the 
school, and that Second Step would be a better fit. Because Second Step is a relatively 
short program, the provider supplemented lessons with Steps to Respect, the companion 
anti-bullying program, and wrote lessons based on the suggested extension lessons. 
Lessons were provided twice a week for 30 minutes each. 


Life Skills Training 


Life Skills Training (LST) is a social skills program that is designed to teach general 
social, self-management, and drug resistance skills. Implementation of the full program 
occurs over a three-year period with fifteen lessons the first year (three of these are 
optional, violence prevention lessons), ten lessons the second (including two optional 
violence lessons), and five lessons plus four optional violence prevention lessons in the 
third year. For maximum effect, all of the core lessons, approximately 45 minutes in 
length, should be taught in sequence. Some of the lessons take two class periods. 
Interactive teaching methods are recommended in order to achieve the full benefit of the 
program. 


Model Program Status 

LST has been designated as a “Best Practice” by the Center for Substance Abuse 
Prevention, an “Exemplary Program” by the U.S. Department of Education-Safe 
Schools and the Office of Juvenile Justice and Delinquency Prevention, and a “Model 
Program” by the National Institute of Drug Abuse. Research on Life Skills Training has 
shown reductions in cigarette smoking of up to 61% (www.lifeskillstraining.com). 
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Implementation 

LST is being implemented as developed at two sites and in a much modified form at a 
third. During the first year of SPIRIT, these three districts implemented the Year 1 
curriculum with all grade levels in the middle school. In each subsequent year, the booster 
years’ curricula have been added as appropriate except at the third site. It has a two year 
junior high and thus can only implement two-thirds of the curriculum. 


Reconnecting Youth 


Description 

Reconnecting Youth (RY) is a program for high school students who demonstrate signs of 
problem behaviors that put them at risk for school dropout. It teaches life skills and 
provides social support as the means to enhancing self-esteem, decision-making, personal 
control, and interpersonal communication and to increasing school-bonding and pro-social 
relationships. The program addresses multiple risk factors including academic failure, 
persistent anti-social behavior, low school bonding, favorable attitudes toward alcohol or 
drug use, and friends involved in problem behaviors. Students may be invited to 
participate in the class if they have a high rate of absenteeism or truancy, have earned 
fewer than average number of credits for their grade level, and/or show signs of problem 
behaviors such as substance abuse, depression or suicidal ideation. 


The program is optimally taught as a daily, semester long, 55-minute Personal Growth 
Class, to a heterogeneous group of students. The program includes suicide prevention 
training for teachers. 


Model Program Status 

RY is designated as a CSAP “Best Practice” and as an “Effective Program” by the 
National Institute of Drug Abuse and the Office of Juvenile Justice and Delinquency 
Prevention. The program has demonstrated significant effects on alcohol and other drug 
use, and on suicidal risk behaviors. It has also been shown to improve school 
performance, decrease deviant peer bonding, and increase self-esteem, personal control, 
and mood management among participants. 


Implementation 

RY was implemented in two districts during the 2002-2003 school year. It was fully 
implemented at one site, but space problems prevented full implementation at a second 
site, and the program was dropped after one semester. RY was added in the alternative 
school at a third site in 2003-2004, but only selected lessons were taught one time per 
week to the three to four students. 
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B. AMOUNT OF PROGRAM CONTACT (DOSAGE) 


A critical element to the effective implementation of evidence-based programs is related 
to program contact. The effectiveness of these evidence-based programs was based 
specifically upon delivering services to youth for a concrete amount of time, and if a 
program veers from this amount, there is a likelihood of reduced effectiveness. Therefore, 
the evaluators of the SPIRIT program have collected data regularly from teachers and 
providers regarding the amount of contact that youth had with the SPIRIT program. 


Figure 4 below presents the variation in the amount of total contact hours by district and 
program. The amount of contact time varied across districts because of the type of 
program offered and differences in implementation. Due to some difficulties in getting 
teacher to complete forms documenting the amount of contact at some of the sites, data 
represent approximations of service contact only. 


Figure 4. Total Contact Hours of SPIRIT by District and Type of Program 
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The program with the greatest amount of program contact was Reconnecting Youth (RY), 
followed by Second Step. These numbers were anticipated because RY is a more 
intensive program than the other four. The amount of time spent participating in Positive 
Action was relatively similar across sites. The figures below indicate the amount of 
program contact at three of the districts for elementary school children by grade.* Contact 
across grades for both years was fairly stable, with much more contact in Year 2 than Year 
1 because of late program start in year one. At Sites A and D, participation was fairly 
consistent across grades, at Site E, there was more variation, with Kindergartners 
receiving the most service in Year One, and 3“ and 5™ graders receiving the most service 
in Year Two. 


Figure 5. Amount of Program Contact by Grade, Site A, Year 1 & 2 
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> One site did not implement a program at the elementary school, and a second did not have reliable data from 
teachers across grades, therefore only data from three sites are presented. 
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Figure 6. Amount of Program Contact by Grade, Site D, Year 1 & 2 
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Figure 7. Amount of Program Contact by Grade, Site E, Year 1 & 2 
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Middle school contact also varied by site, with fewer hours of implementation in Year 
Two than Year One across three sites (Figure 8 below). This information is complicated 
by two factors. LST provides fewer lessons in each of the three years covered by the 
curriculum. During the first year of implementation, the lengthiest segment was taught to 
all students. By year 2, older students were being taught fewer lessons. The total time 
spent on lessons was averaged across all students, causing it to appear as if the dosage was 
lower. Additionally, during year 2, at one of the sites, only four teachers implemented a 
SPIRIT program. 


Figure 8. Amount of Program Contact in Middle School by Site, Year 1 & 2 
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C. PROGRAM ADAPTATION 


Another measure of fidelity is the degree to which programs maintain the original 
program design. Some sites will implement a program exactly as proscribed by the model 
developers, others will make significant modifications to meet local needs. Because of 
the nature of SPIRIT, the chosen evidence-based programs are being implemented with 
various adaptations throughout the five districts. During the project, changes have been 
made to programs for such reasons as to fit them into the allotted class time available, to 
make them more culturally sensitive, and to make them more age appropriate. Because of 
this, the evaluation has had many fidelity issues with which to contend. 


One of the most dramatic examples of program modification is that an entire curriculum 
was written and rewritten for Positive Action at the high school level. Not only was this 
curriculum incomplete when it came from the developer, but it was not considered to be 
age or culturally appropriate. While much of this modification was done in collaboration 
with the program developers, this is the first real evaluation of the curriculum. Another 

adaptation made at several districts related to the amount of the curriculum provided due 
to constraints on available classroom time. 


D. CONCLUSION 


SPIRIT includes a strong cadre of evidence-based programs, including programs geared to 
the general population of students and programs targeted specifically for youth who are at 
risk or already indicating problems. SPIRIT was designed to test the results of 
implementing evidence-based prevention programs from grades kindergarten through 12, 
and data indicate that programs are serving children across this continuum in most 
districts. 


Adaptations to programs have been made across the five participating sites to 
accommodate local circumstances and the students being served. In the case of one 
program, an entire curriculum has been re-vamped by providers. Other programs were 
modified because of time limitations, and others replaced one program with another. 
These adaptations point to the need for early communication with schools, teacher 
training, and pilot testing of curricula prior to full program implementation. When these 
preparatory steps are taken, program fidelity and ultimately program effectiveness are 
improved. 
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CHAPTER THREE 
SITE VISIT AND FOCUS GROUP FINDINGS 


In May and June of 2004, site visits were made to all five districts to provide first hand 
information about program implementation, teacher attitudes, program strengths and 
weaknesses, and total impact of the prevention programs. The site visits also assessed 
student response to the SPIRIT program. At each site, we observed and documented 
evidence of the SPIRIT program throughout participating schools. In addition, we 
conducted focus groups with teachers, and in some instances, had discussions with 
providers and principals as well. A total of 95 individuals participated in focus groups 
and interviews. In all districts, teachers, principals and providers were welcoming and 
open in response to our questions. The qualitative data provided through observation, 
conversation, and focus groups provides a context in which to interpret the quantitative 
data, and is important in evaluating the effectiveness of SPIRIT. 


All participating individuals were asked to discuss their impression of SPIRIT, including 
what they felt went right, what they felt could be improved, what obstacles they 
encountered, and what level of parental involvement was present. More specifically, 
individuals were asked what specific changes they saw in the school climate, the 
behavior of the individual children, and parental involvement in school as a result of 
SPIRIT. 


A. POSITIVE FEEDBACK REGARDING SPIRIT PROGRAM 


SPIRIT received positive feedback in many areas including 1) links between providers 
and teachers, 2) improved student behaviors (both immediate and long term), 3) 
improved school climate, 4) indirect effects on pre-school children, 5) effects on special 
populations, 6) reductions in disciplinary actions, 7) effects on other academic areas, and 
8) changes in school climate. 


Provider-Teacher Linkages 
One of the major themes that emerged during focus groups was the importance of the 
link between the schools and provider agencies. This was summed up by one teacher: 


“Don’t take the SPIRIT guys away from us!” A principal from another district talked 
about the importance of having the providers in school as part of the school community. 


“Having someone in the building so that the program was not just thrown on us was 
important. We wanted someone who would participate and have consistency and follow- 
through. [The prevention specialist] has created good rapport. The staff has accepted 
her because she is willing to do many things. She has helped to adapt the curriculum 
when changes were needed, and she has served on the SAFE team (student assistance 
team), something few teachers like to do.” 


The principal also stated that it would have been helpful if they had known more in 
advance, and wondered about what would happen when SPIRIT ends. 
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Agencies have provided a range of site-specific services, and in all instances, have 
reportedly given stability to the programs and the children through classroom and 
curriculum support. “The kids are excited when [the provider] comes in. He always 
portrays the positive. He is a good male role model and mentor, especially for kids who 
don’t have one.” Teachers also reported that the providers 1) trained teachers in the 
program, 2) modeled teaching methods, 3) produced special events for the students (and 
sometimes parents as well), 4) assisted with specific student needs, and 5) provided 
support and encouragement to teachers, administrators and students. Following are some 
of the comments made about provider assistance. 


“..we can do a lot of talking, but they [the children] have to transfer it [the program] 
to real life. Having [the provider] to help has meant that kids have remembrance of how 
things actually happened with the help of the “PeaceBuilder lady.” They are “applying 
PeaceBuilders” instead of just role-playing.” 


“[The prevention specialist] is a wonderful asset. She has been instrumental in helping 
teachers—she works with the kids to help them understand the importance of education. 
She is someone who cares.” 


“We couldn ’t have better support than [the prevention coordinator]. It is great to have 
her in the building.” 


“[The providers] are awesome. They check with us to see if there is anything we need.” 


In sum, teachers felt that providers were a great asset to their schools because they 
provided a type of support not previously available. 


Immediate Changes in Child Behaviors 

In addition to giving support to teachers and schools, focus group respondents believed 
that the programs have resulted in behavioral change in children. While teachers, 
administrators and providers were eager to know more about the statistical results of 
SPIRIT, they all expressed the belief that there are differences in participating children 
and youth as a result of the program. At each site, we heard stories about the effects on 
individual students and the school community. We were given copies of letters from 
Second Step students thanking their teachers for teaching them, through role-play, how 
to handle difficult situations. We were touched to meet an RY student, once considered 
“high risk,” who had been selected to attend a national leadership conference in 
Washington D.C., and who now is providing inspiration and direct assistance to other 
“high risk” students. 


Teacher comments included the following: 


“I don’t know whether differences are because they are growing up or because of the 
program, but with some kids, there are night and day differences.” 


“T see differences in students this year who had it last year.” 
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“Third graders are very receptive. As the year progresses, they show more respect, do 
less arguing, and on the playground, some kids will intervene in fights and use 
PeaceBuilders to help solve the problem.” 


“Kids say it makes them think about what’s right and what’s wrong.” 


“Kids love the smoking and biofeedback sections [Life Skills Training]— Kids also love 
the advertising section and are surprised to learn that advertisers target particular age 
groups with their ads. They enjoyed a competition to write advertisements in Language 
Arts for local businesses.” 

Teachers reported “much less problem with kids who won’t own up to their behavior.” 
“They take responsibility easier than they used to. Kids are doing stuff wrong, but they 
know it, they can admit it, and they are learning to solve it.” One example was of a 
student who cheated on running laps, got her card punched, and the next day, 
acknowledged that she had cheated, and willingly ran extra laps to make up for the ones 
skipped. [PeaceBuilders] 


Ultimate Outcomes 

Some teachers and principals expressed confidence that ultimately, there will be positive 
outcomes. “Jt takes time to show results. After next year, we'll see differences.” “After 
only two years it is hard to say. Maybe after three or five.” A principal stated that he 
believes “as kids come up” who have had the program from a young age, “we will see 
more results.” One teacher (grade 6) reported that some students had said after lessons 
they wished that they had discussed issues with an adult before making decisions. 


It was reportedly harder to engage older students than younger ones. “Juniors and 
seniors are the hardest ages to interest.” Teachers wondered, however, if there would 
be a different experience with children and youth after a couple of years of experience 
with program participation. 


Indirect Effects 

In addition to affecting children and youth in school, we also learned about how 
PeaceBuilders was transferred to pre-school siblings and how students challenge others 
to be “PeaceBuilders.” 


“Pre-schoolers are using the language—they must be getting it from their older brothers 
and sisters.” One parent added, “ J have a preschooler at home who brings what she 
learns in PeaceBuilders with her. I know that at my home, it never stops arguments but 
sometimes I notice that it makes them think about things. I notice that a sort of ‘internal 
dialogue’ of ‘what’s the best course of action to follow is going on.” 


The programs have provided the means to recognize students for positive behavior. 


“It gives ‘good kids’ something to bank on: ‘I’m a PeaceBuilder.’ They have a name for 
themselves.” 


“Now we notice when someone is a PeaceBuilder—we notice the positives that 
otherwise we wouldn’t. That is true for both teachers and students. Positives are written 
up in praise notes.” 


“We used to have ‘good’ students, and it was limited to one or two. Now we have 
‘positive students’ and it is unlimited.” 


Special Populations 

The programs have also supported inclusion of students with identified problems and 
differences. We were told that learning to “sign” the PeaceBuilder song has given 
children a greater connection to a hearing impaired child in the school. Children also try 
to sign other things to hearing impaired students. A special education teacher said, “Tt is 
easier for everybody because the whole school is working on the same thing. It has had 
a big impact on the social relationships of students with special education.” Through 
RY, participants became involved in political action and produced meaningful letters to 
the editors in support of a school levy that would save their school. Positive Action 
helped to support a group of abused middle-school girls and helped them to deal with 
the conflict in their lives. These are all meaningful successes that are hard to capture in 
statistical analyses. 


Other Academic Areas 
The effects of the programs have also transferred to other curricular areas. 
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“Tt shows up in kid’s writing. They know more positive words to use in descriptions.’ 


Discipline 
Staff reported positive changes in discipline and associated problems. 


“It seemed like when the Care Team began, half the kids in school were being referred. 
Now there are fewer. Teaching PeaceBuilders and role-playing has brought different 


a” 


responses. 
“The secretary would say that discipline referrals have dropped.” 


“When kids are arguing and are asked about PeaceBuilder ways, there is a pause. They 
know—they might not always use what they know, but they know.” 


Changes in School Climate 

Improvement in school climate is an important measure of program effectiveness for 
some of the prevention programs, particularly PeaceBuilders and Positive Action. 
Positive school climate is reflected in increased school bonding, a protective factor that 
tends to help improve academic performance, one of the major goals of SPIRIT. 


Teachers talked about how programs gave a sense of identity to students and schools. 
“PeaceBuilders is a constant for the kids. They may change grades and teachers, but 


PeaceBuilders remains constant. It gives an identity to the whole school—We are 
PeaceBuilders.”’ 
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“Students don’t view it [the program] any particular way. They see it as a way of life at 
school. They don’t realize that other schools don’t do PeaceBuilders.” 


Having commonly understood language was seen as helpful. Any time the question “Is 
that being a PeaceBuilder?” was asked, students and teachers alike understood what that 
meant. Likewise, RY students started working with other students to improve 
attendance. For example, they would call a tardy or absent student and tell him/her “you 
need to get here.” 


“Kids notice the behavior of other kids. They are learning the limits of their control, but 
they are doing everything they can. Kids know that they are being watched, and they 
know it’s because others care.” 


B. OBSTACLES TO IMPLEMENTING THE SPIRIT PROGRAM 


Several themes emerged in response to the question, “What obstacles or road blocks 
have you encountered. What went wrong?” These included 1) lack of teacher buy-in; 2) 
lack of time to fully implement the program and to prepare to deliver the curriculum; 3) 
little consistency across teachers in implementing the program; 4) lack of 
communication between teachers and providers; 5) specific issues with curricula, e.g., 
age- and cultural-appropriateness; 6) initial misunderstanding of SPIRIT purposes; and 
7) lack of continuity. 


Teacher ‘buy-in’ 

Getting buy-in from schools that have had experiences with externally funded programs 
that leave after a few years of implementation was difficult. We learned from a 
principal that their school had been informed by the district that the program they had 
chosen was not the one they would be implementing. There was, therefore, some 
distrust. “/The prevention specialist] was helpful in smoothing things out. She gained 
the respect of teachers with the in-service she conducted, and she put teachers at ease. 
Teachers have been more receptive, but it is always hard when something new is 
added.” 


Time 
Prior to SPIRIT implementation, teachers had a full schedule. Determining how SPIRIT 
could fit into regular school year schedules was, therefore, difficult. 


“We are loaded, time-wise. It is difficult to work in.” 


“Tuse ‘teachable moments,’ but it is hard to find time for all the lessons I would like 
to.” 


Because lessons seem to help reduce pressure and ease tensions, sometimes teachers opt 
to include them even during heavy academic times. 


“When the academic requirements increase, we have to stop the lessons, and the kids 
know there is more pressure. There are times when I choose to do a [PA] lesson 
anyway.” 
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“It provides a time without the pressures of teaching—just ‘friends and family’ talk 
about life. It lets us take a step back, breathe and just talk.” 


Teachers also agreed that they needed preparation time to review lesson plans prior to 
program implementation. A comment that elicited general agreement was that “getting 
the curriculum over the summer so that I would have had more time to look at it” would 
be helpful. 


Lack of consistency in implementation 
Other obstacles were that some of the programs are taught for one semester only, or that 
some teachers opt not to teach or reinforce the program. 


“The only thing would be that it’s only targeting certain kids. Most teachers don’t 
implement. Kids need to be referred to Positive Action.” 


“PeaceBuilders is great. But there isn’t anything like it in the high school, the 
“SPIRIT” thing. It is all restricted to classrooms. It’s inconsistent. Maybe if other 
teachers could reinforce it, it would help. They walk out of the classroom, and it is 
done.” 


Administrative support was thought to be important in dealing with these problems. 


“It would help to have administrative and staff support. Because [the prevention 
program] is not being implemented school-wide, we don’t have exchange.” 


Specific curriculum issues 
Teachers identified problems with the Positive Action curricula, particularly at the 
middle and high school levels. Comments included: 


“The curriculum [Positive Action] is dated. The copyright says 1987. Some of the 
characters in the stories are unfamiliar to today’s students.” [Grade 7] 


“Positive Action is not a High School curriculum—it is not age appropriate. Other 
curricula are better. It was promised to be in better form . . . It is irritating to have so 
much money spent on a poor curriculum.” [Grade 9] 


These problems, however, were not seen at all districts using Positive Action in these 
grade levels. A teacher from another district said, “We didn’t have a problem with 
inappropriate materials. All kids are going through the same things. If there were 
problems, we tweaked it a little. Positive Action touches on kids’ issues.” 


The elementary level version of Positive Action was dropped in one district because 
teachers and providers thought it was not culturally appropriate, and did not serve the 
primary needs of the school. Second Step was chosen to replace it because of its focus 
on violence prevention. Teachers were pleased with the change. They said, “ J think 
students are part of the discussion this year. This year is definitely better.” “TI think 


most problems were fixed when they changed the curriculum.” “The role plays are more 
realistic; very different from the past. I think it is more age appropriate.” 
Misunderstanding Regarding Program Purposes 


Continuity 

Teachers thought that prevention class schedules had an impact on continuity when 
lessons were given too far apart. “J don’t like the schedules of the prevention classes, 
there’s too much time between lectures.” “We should do it [Positive Action lessons] 
every day or not at all. Doing it once every two weeks is less than DARE... The kids 
enjoy the stories and they build on each other. We need to do them more often.” 


Giving a program lesson every day was seen as beneficial. Daily lessons have allowed 
teachers to have ongoing discussions. “We do it every day, and it makes the kids more 
relaxed.” 


Communication Between Teachers and Providers 

In one district where the providers teach lessons, the majority of comments about the 
providers and program were positive. It was noted at all levels, however, that more 
communication between classroom teachers and prevention specialists would be 
beneficial. Teachers also felt that a better understanding of the normal program 
curriculum (i.e., wellness classes) would enable providers to better tie SPIRIT into 
existing programming. 


C. SUMMARY 


SPIRIT was very positively received across the five participating districts. It was 
apparent that the creativity and commitment of teachers and providers is what makes the 
program work. Providers have sponsored special events, like “Donuts for Dads” and 
“Muffins for Moms,” produced newsletters for parents and teachers, and held field days 
and kick-off ceremonies. Teachers have found ways every day to reinforce the 
programs, using techniques such as marble jars that count positive actions and posters 
honoring “wise people” in the school community. In the kindergarten of one school, the 
teacher sent home a note asking how the child was special. She then recorded the 
responses, and played the recordings for the children. The efforts of these committed 
teachers and providers have given the real “spirit” to SPIRIT. One teacher summed it up 
this way: “Whether they admit it or not, parents have the greatest influence, but I hope 
that we add something.” 
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CHAPTER FOUR 
OUTCOME FINDINGS 


A. INTRODUCTION 


A major objective of the evaluation was to analyze change in behaviors among children 
and youth involved in SPIRIT as they progressed through the prevention programs. This 
chapter describes an on-going sample of youth participating in SPIRIT and contains 
findings for those youth who completed questionnaires at all four time points (Fall 2002, 
Spring 2003, Fall 2003, and Spring 2004).* The distribution by school district is 
presented in the chart below. 


Figure 9. Number of Youth Completing Questionnaires At All Time Points 
in Each District 
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Three different instruments were used to measure how well children and youth were 
progressing as a result of SPIRIT. Children in grades K-3 were assessed by teachers 
using a form that measured changes in aggression and social skills. Students in 4" and 
5" grades completed a survey that assessed risk and protective factors related to 
adolescent substance use. Students in 6" through 12" grade completed a survey 


* Prevention programming at the schools may have been present prior to Fall 2002; SPIRIT started in 
Fall 2002. 
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measuring substance use, its perceived harm, individual risk and protective factors, anti- 
social behaviors, and family management. Additional data collected on each individual 
student from 4" through 12" grade included grades, disciplinary incidents, school 
attendance, race, age, and gender. This chapter discusses changes over two years and 
four testing times. 


Repeated measures Analysis of Covariance (ANCOVA) was used as the statistical 
method to measure change over time. Only those youth who assented to participate and 
who had parental consent were included in the evaluation. There were 1,238 surveys 
for Time 1; 1,141 surveys for Time 2; 2,040 surveys for Time 3; and 1,934 surveys for 
Time 4. To provide an analysis of change over time, only students who completed the 
surveys across all 4 time points were included in this analysis. 


B. DEMOGRAPHIC CHARACTERISTICS OF THE TOTAL SAMPLE 


Students participating in SPIRIT varied widely in age, racial/ethnicity, and gender. 
Their ages ranged from 6.7 years to 20.6 years at Time 4. The sample was almost 
equally male (48%) and female (52%). The largest percentage of participating students 
were white (53%), and African-American (41%), with a small percentage of Hispanics 
(4%). The sample was fairly evenly divided by school level, with 35% in elementary 
school, 37% in middle school, and 28% in high school. 


C. OUTCOME FINDINGS 


Kindergarten — 3 Grade 


Because of the age of students, Kindergarten through 3" grade SPIRIT participants were 
assessed by teachers for aggressive behavior and social competence, two factors 
considered precursors to substance use, using an instrument called the Teacher 
Observation Checklist. The students (n = 137) included 49 Kindergartners, 57 1* 
Graders, and 31 2" Graders (at Time 1). Third graders were not included in this 
analysis because of their transition to fourth grade during Year Two and therefore 
transitioned to a different survey instrument. Students remaining in the sample 
represented four of the five school districts participating in SPIRIT. The majority of 
students were African American (73 %), and Caucasian (24.8%). Students were either 
participating in Positive Action (80.3%) or Peace Builders (19.7%) in the first year of 
the project. In the second year of the project, however, program changes were made and 
students were participating in one of three programs: Positive Action (13.9%), Peace 
Builders (19.7%), or Second Step (66.4%). 


Figure 10 shows teachers’ perceptions of aggressive behavior among SPIRIT children at 
times 2 and 4 (Spring 2003 and 2004).° Reactive aggression (self-defense) decreased 
from the first to the second year, while proactive aggression (using physical force to get 


> Data from Times 1 and 3 were not presented. Teacher reports of children’s behavior are more accurate at the 
end of a school year than at the beginning of the year because they have had an opportunity to observe and work 
with children for a period of time. 
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what you want) remained the same. This is a very encouraging finding. Overall, it is 
very difficult to change aggressive tendencies in children (Aber, Brown, & Jones, 2003). 
The literature on aggression in younger elementary school students indicates that 
proactive aggression is harder to change than reactive aggression because it requires 
both parent/teacher interventions as well as child interventions (putting a child in time 
out, etc.). Reactive aggression can be addressed through individual interventions, 


especially teaching children self-control. 


Figure 10. Change in Aggression Over Time, Spring 2002 and Spring 2003 
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Changes in aggression over time were more pronounced among females than males. 
While females were less aggressive than males at the end of the first year, they 
continued to become less aggressive, whereas males decreased only slightly. Again, 
these decreases are encouraging because aggression is hard to affect with this age group. 
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Figure 11. Aggression Over Time by Gender, Spring 2002 and Spring 2003 
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When we divided children into groups composed of those who were assessed with low 
and those with high levels of aggression and analyzed them separately, we found the 
same pattern as with the total sample. Proactive aggression did not change for either 
children with low or children with high levels of aggression, whereas reactive 
aggression decreased slightly for both groups. 


Figure 12. High and Low Risk Children, K-3, Times 2 & 4 
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Social Competence 

The Teacher Observation Form has two scales, called Prosocial Behavior and Emotion 
Regulation, that measure social competence. The Social Competence Scale measures 
teachers’ reports of whether a child engages in certain prosocial behaviors and how well a 
child controls his or her emotions through two sub-scales: Prosocial Behavior and Emotion 
Regulation. This scale consists of 19 total items rated on a 5-point scale (1 = not at all; 5 = 
very well). The Emotion Regulation sub-scale consists of 8 items (i.e., “Copes well with 
failure”; “Can wait in line patiently when necessary”). The Prosocial Behavior sub-scale 


consists of 11 items (i.e., “Acts friendly toward others”; “Listens to others’ points of view”). 


As was anticipated because of the maturational trends for girls and boys of this age, 
teachers reported that females exhibited a higher level of Prosocial Behavior and 
Emotion Regulation than males at time 4 (Figures 13 and 14). There were no significant 
decreases in social competence over time, indicating that the programs were successful 
in maintaining social competence as students matured. These patterns were similar for 
children when high and low emotion regulation and high and low Prosocial Behavior 
were measured separately. As with aggression, these behaviors are very difficult to 
improve in younger children (see Springer et al., 2002). 


Figure 13. Teacher Observations of Students’ Levels of Emotion 
Regulation 
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The lack of increase in social competence, therefore, indicates programs were able to at 
least maintain the level of social competence, which can decrease as young children grow 
older (See Figure below). 
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Figure 14. Teacher Observations of Social Competence 
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Absences and Disciplinary Actions 

For students participating in the evaluation, schools were asked to provide information 
regarding the number of times that students were absent and the number of times that a 
child received a disciplinary action. Unfortunately, the data are not reliable because of 
the way schools report absences. Both excused and unexcused absences are kept on 
school records. Absences, therefore, would not show problem behaviors, but might 
reflect illness or an excused activity. The number of disciplinary incidents across the 
participating schools was negligible during both years, indicating that for these grades, 
discipline is not yet problematic. 


Outcome Findings: 4th — 5th Grades 


Students (n = 84) in the 4" and 5" Grades completed the Healthy Kids Survey at four 
administration points from Fall 2002 through Spring 2004. The Healthy Kids Survey 
assesses substance use and perceived risk of substance use, empathy, problem solving, 
and school, home and peer environment. 


There were 76 4"" Graders, and 8 5" Graders who completed the survey at all four 
points. These students were in four of the five school districts participating in SPIRIT. 
There were a relatively equal number of males and females: 46 (54.8%) females and 38 
(45.2%) males. Students ranged in age from 9.2 to 11.5 years of age (at Time 1). 
African American (47.6%), and Caucasian (48.8%) students were equally represented. 


Substance Use and Harm 

Students were asked how frequently they used cigarettes and alcohol in the month 
previous to testing. Use rates were extremely low at all time points. Although the 
percentage of use increased slightly, there were no significant differences between Years 
One and Two. 


Figure 15. Use Rates of Cigarettes and Alcohol, 4™ & 5" Grades, Times 1 & 4 
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Students were asked whether they thought cigarettes, alcohol, and marijuana were bad 
for their health. The majority of students thought using these drugs was “very bad” for 
one’s health at the first data collection point, indicating that messages regarding the 
harmful effects of cigarettes, alcohol and marijuana had already been taught to the 
children either at home or at the schools (Figure 16). Across time, the pattern of 
responses toward marijuana use remained the same, with about 80% of the students 
indicating that marijuana use was “very bad.” Cigarette, and especially alcohol use, 
became more acceptable as the students got older, reflecting an environment that still 
considers cigarette and alcohol use acceptable. Nevertheless, 80% of students at Time 4 
still thought cigarette use was “very bad.” These results mean that prevention programs 
need to work further to impress upon children the dangers related to alcohol abuse and 
tobacco addiction. 
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Figure 16. Perceptions of Alcohol and Cigarette Use, 4 & 5™ Grades, 
Times 1 & 4 
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In summary, among 4" and 5" graders, cigarettes and alcohol were used only among a 
very small minority of students, and rates of use did not change from year to year. 
Perceptions of harm from cigarettes, alcohol, and marijuana were very high at the 
beginning of Year One. Cigarettes and alcohol were perceived to be less harmful by the 
end of Year Two when students were in 5" grade. Marijuana, the one illegal drug 
mentioned, was still perceived to be very harmful by Time 4. These data suggest that 
programs need to stress the dangers of cigarettes and alcohol more to combat the social 
norms associated with these legal substances. 


Anti-Social Behavior 

Hitting, kicking, verbal insults and threats among elementary school children are all 
anti-social behaviors that may potentially escalate into more serious behaviors 
(homicide, assault) in adolescence (Conduct Problems Prevention Research Group, 
1999; Dahlberg, 1998). Curbing anti-social behavior is therefore extremely important. 
These behaviors, however, are some of the most difficult to affect among elementary 
school children. Several studies have shown that changing anti-social behaviors require 
intensive interventions that involve students, parents, and teachers (Flannery et al., 2003; 
Farrell & Meyer, 1008; Gottfredson, 1997). Other studies of aggression among this age 
group have shown natural increases in physical aggression without the aid of intensive 
interventions (Grossman et al., 1997). A very challenging task of the SPIRIT program is 
to affect anti-social behaviors in elementary school children in order to prevent a 
negative developmental trajectory from occurring. 
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SPIRIT students were asked to indicate the frequency of negative (delinquent) behaviors 
over the past year such as hitting and pushing other students, spreading rumors or lies 
about other children, and bringing a gun or knife to school. Among 4" graders who 
started the program in 2002, the majority indicated that they had not brought a gun or 
knife to school. There was no significant change in their report of this behavior over 
time. Hitting and pushing were more common, increasing from Time | to Time 4, when 
60% of all youth in the sample reported hitting or pushing. Spreading rumors, a form of 
aggression attributed to females, was minimal at Time | and increased over time. As 
expected, rumor spreading was higher for females both at Time | and Time 4 (Figure 
18). 


Figure 17. Percentage of Youth Never Engaged in “Delinquent” Behavior, 
4" & 5th Grades 
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Figure 18. Gender Differences in Rumor Behavior 
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Participants were also asked if other students ever hit or pushed them or told rumors or 
lies about them (Figure 19). The majority of students indicated that either they were hit 
or pushed never (44% - 51.2%) or only some of the time (23.8 — 33.3%). These rates 
were not different from each other at any time point. Students also indicated that they 
either never (28.6 — 53.6%) or rarely (29.8 — 42.9%) had rumors spread about them 
(Figure 19). 


Figure 19. Percentage of Students Never the Subject of Delinquent Behavior 
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Importantly, Second Step, with only one year of implementation, has shown reductions 
in aggression. The sample size for that program is too small to report here, although 
focus group findings suggest a positive impact. These findings, as well as those 
presented in the figures below, stress the need for interventions that directly address how 
harmful hitting, pushing, and spreading rumors are, and the implications of these 
behaviors for youth as they get older. 


Disciplinary Actions 

Schools were asked to provide the number of students’ disciplinary actions. The 
average number of incidences decreased from .63 in Year | to .49 in Year 2, but these 
differences were not statistically significant. 


Figure 20. Average Number of Disciplinary Incidents, 4 and 5" Grades, 
Year 1 & Year 2 
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Figure 21 shows that student perceptions of school environment decreased over time. 
This is similar to attitudes found among children in this age group. Because students’ 
perceptions of their school environment naturally decline, this result may not reflect be a 
negative consequence of the program. It may, however, suggest that greater effort is 
need to counter this trend and enhance the school environment. 


Figure 21. Students’ Rating of School and Peer Environment °, 
4" and 5" Grades 
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* A high score indicates a more positive perception of the environment. 


For those students who perceived their environment negatively at the beginning of Year 
1, however, perceptions of the school environment did not decrease but remained stable. 
This suggests that programs were successful in preventing a greater decline in the 
perception of the school environment among this group of students (Figure 22). 


° The School Environment scale consisted of 4 items (i.e., “Do the teachers and other grown-ups at school care about 
you”; “Do the teachers and other grown-ups at school tell you when you do a good job”; “Do the teachers and other 
grown-ups at school listen when you have something to say”; “Do the teachers and other grown-ups at school believe 
that you can do a good job”). The Peer Environment Scale consists of 2 items (i.e., “Do your best friends get into 


trouble’; “Do your best friends try to do the right thing”). 
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Figure 22. Average School Environment Rating of High and Low Risk Students 
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* A high score indicates a more positive perception of the environment. 


Fourth and fifth graders were asked about their empathic abilities and problem solving 
skills. Students with lower empathy and lower problem solving skills at the beginning 
of the evaluation remained essentially the same over time, while students with higher 
empathy and problem solving skills declined slightly over time, reflecting natural 
maturational trends. 


Figure 23. Average Empathy/Problem Solving Rating of High and Low Risk Students 
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Outcome Findings: 6th — 12th Grades 


Students in the 6" through 12" grades (n =396) completed the SPIRIT Survey at the four 
administration points from Fall 2002 through Spring 2004. The SPIRIT Survey 
measures substance use, family management, stress management, decision making, self- 
esteem, perceived risk of using substances, frequency of anti-social behavior, and 
attitudes toward substance use. 


Substance Use 

Substance use among 6-8" graders was relatively low at program entry (Figure 24). At 
both times, alcohol was the most commonly used substance, followed by cigarettes. 
While use increased for all substances from the first to the fourth time point, change was 
greatest for alcohol use. 


Figure 24. Percentage of Youth Reporting Substance Use in the Past Month, 
6" — 8 Grades 
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For high school students (grades 9 through 12) in the sample, marijuana and inhalant use 
declined from Time | to Time 4, alcohol use did not increase significantly, and only 
cigarette use and binge drinking continued to climb. Given that patterns of substance 
use tend to increase as adolescents continue to mature, these patterns are very 
encouraging. Also encouraging is the fact that inhalant use decreased, contrary to 
national trends. 


44 


Figure 25. Percentage of Youth Reporting Substance Use in the Past Month, 
9" 12" Grades 
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To place these results in context, comparisons of substance use were made between 
SPIRIT participants and a national sample of youth at ages 13 and 15. This comparison 
shows that while at 13 years old, when SPIRIT youth entered the program, the 
percentage of youth smoking cigarettes, binge drinking and using marijuana was 
approximately that of a national sample of youth, after year two, the percentage of youth 
using was significantly less than the national sample (Figure 26). 
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Figure 26. SPIRIT Substance Use Compared with a National Sample’ 
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In summary, alcohol is the most prevalent drug of choice among this age group, with 
fairly significant binge drinking and cigarette use. Reported illegal drug use, including 
marijuana, is low. Comparisons of use rates with national data are very encouraging. 


Rates of cigarette, alcohol, and marijuana use among SPIRIT youth are significantly 
lower than rates of substance use for this age group. 


Risk and Protective Factors 


Figure 27 displays findings between Time | and Time 4 for risk and protective factors 


and attitudes toward substance use. Risk and protective factors included family 
management practices, decision making, stress management, self-esteem, school 


commitment, perceived risk of drug use, frequency of anti-social behavior, and attitudes 


toward drug use. 


The two areas in which positive change occurred were anti-social behaviors and 


decision-making. The frequency of antisocial behavior declined slightly, and decision 
making improved slightly. Family management, stress management, and self-esteem 


remained the same. School commitment, perceived risk of drug use, and attitudes 
toward drugs and antisocial behaviors all declined. 


7 National data are from the National Survey on Drug Use and Health (NSDUH), 2003 
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Figure 27. Average Selected Risk and Protective Factors Over Time 
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Anti-social Behaviors 

The frequency of anti-social behaviors decreased overall between Time | and Time 4. 
To learn more about this, an analysis of each item was conducted separately for middle 
school and high school (6"" through 8" grade and 9" through 12" grade). Results show 
that bullying and physical aggression decreased over time for 6th-8" grades. Being 
threatened by a weapon increased slightly (Figure 28). 


Figure 28. Percentage of Students Reporting Bullying and Violence, 
Over Time, 6" and 8" Grades 
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In the 9" through 12" grade group, bullying, physical aggression and being threatened or 
injured with a weapon all decreased over time. The largest percent decrease was in 
bullying between Time 1 to Time 4 (Figure 29). 


Figure 29. Percentage of Students Reporting Bullying and Violence, 
Over Time, 9" — 12" Grades 
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Antisocial and Delinquent Behavior 

There were very low rates in arrests, use of weapons, and suspensions over time among 
6"" through 8" graders, selling of illegal drugs, stealing or attempts to steal a motor 
vehicle, or being drunk or high at school. While some increases in these behaviors exist, 
the rates remained very low. School suspensions had higher rates that increased over 
time (Figure 30). 
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Figure 30. Past Year Antisocial and Delinquent Behavior, 6" — 8 Grades 
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Disciplinary Incidents 


There were no statistically significant differences in disciplinary incidents over time 
among 6'" to 8"" graders across time points. Disciplinary incidents declined slightly, 
though not significantly, among 9" to 12" graders. High school students reported little or 
no increases in antisocial and delinquent behaviors. 


Figure 31. Average Number of Disciplinary Incidents Over Time 
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Grades 


For both middle and high school students, grade point averages increased slightly from 


Year 1 to Year 2. These differences were not statistically significant but suggest a pattern 


of change in the intended direction. 


Figure 32. Average Cumulative GPA By Grade Level 
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CHAPTER FIVE 
SUMMARY 








Kindergarten — 3"! Grade Sample 


The findings for the kindergarten through 3™ grade sample suggest that SPIRIT is 
beginning to affect aggressive behaviors among males and females, and emotion 
regulation in females. The data show slight decreases in reactive aggression, which is 
very encouraging given that overall, it is very difficult to change aggressive tendencies in 
children (Aber, Brown, & Jones, 2003). It is expected that proactive aggression was not 
affected because the literature on younger elementary school students shows that this type 
of aggression is harder to change than reactive aggression. Proactive aggression requires 
both parent/teacher and child interventions (putting a child in time out, etc.). Reactive 
aggression is more easily addressed through individual interventions, especially teaching 
children self-control. 


4" — 5" Grade Sample 


Substance Use 

Programs need to increase their efforts to impress upon children the significant problems 
that occur related to alcohol abuse and tobacco addiction. Data show slight increases in 
cigarette and alcohol use from the beginning of the first year to the end of the second. 
Further they show reductions in the children’s perceptions that cigarettes and alcohol are 
harmful. As children develop into pre-adolescence, they also change their models from 
their parents to their peers. Exposure to peers who are using substances, thus, will begin 
to change their perceptions of the dangers of cigarettes and alcohol. 


Anti-Social Behavior 

One of the most difficult behaviors to affect among elementary school children is anti- 
social behavior. Several studies have shown that changing anti-social behaviors requires 
intensive interventions involving students, parents, and teachers. While SPIRIT programs 
in the elementary schools work to create more positive school climates, both 
PeaceBuilders and Positive Action, the primary interventions, may not provide the level 
of intensity a child at high risk requires to reverse the natural maturational trend toward 
more difficult behaviors. Other studies of aggression among this age group have shown 
natural increases in physical aggression without the aid of intensive interventions 
(Grossman et al., 1997). The Second Step program, added in the second year, has shown 
significant reductions in aggression over time; the sample size for that program is too 
small to report here though focus group findings suggest a positive impact. In addition, 
more focused interventions stressing the importance of the harm of rumors are 
recommended, especially for young girls. 


St 


School Environment 

Students’ perceptions of the school environment deteriorated. This may not reflect a 
negative program finding because it is a normal occurrence at this age. It may, however, 
suggest that greater effort needs to be expended by programs to enhance the school 
environment. Changing the school climate often requires many years of focused effort by 
school administrators. SPIRIT’s influence, as evidenced in the focus group data, has 
been very positive, but continued attempts to improve school climate should be a priority. 
The fact that perceptions of the school environment did not decrease but remained stable 
suggests that programs were successful in preventing the natural trend toward a more 
negative perception among this group of students. 


6" — 12 Grade Sample 


Substance Use 

During the middle school years, adolescent substance use begins. Between the ages of 12 
and 15, substance use tends to move from a rare occurrence to a fairly frequent behavior. 
Importantly, SPIRIT was very successful in reducing the rate of increase in substance use 
among middle school students. Findings for high school students are also very 
encouraging showing only a slight increase in alcohol use, binge drinking and cigarette 
use, and reductions in marijuana and inhalant use, between Years 1 and 2. Subsequent 
reports will explore program-specific findings in greater detail to determine whether the 
lack of expected increase is more attributable to a particular program or whether all are 
equally effective to some degree. 


Grades 

There were slight improvements in grades from Year 1 to Year 2 in the sample of SPIRIT 
youth both at the middle and the high school, though these changes were not statistically 
significant. 


Anti-Social Behaviors 

Middle school students showed decreases in the amount of bullying between Fall 2002 
and Spring 2004. They also showed reductions in the amount of pushing, shoving and 
hitting. These results are very encouraging given that students usually increase these 
behaviors in middle school. 


High school students also showed reductions in bullying behaviors between Fall 2002 and 
Spring 2004. Pushing and shoving also decreased for this age group. 


Disciplinary Incidents. There were no statistically significant differences in disciplinary 
incidents among 6" to 8" graders. Disciplinary incidents declined slightly, though not 
significantly, among 9" to 12" graders, again perhaps reflecting the influence of the more 
intensive Reconnecting Youth program in two of the school districts. 
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Summary 


The SPIRIT project is an ambitious attempt to bring evidence-based programs to five 
Missouri school districts. Although caution must be taken in interpreting these results 
because of the small sample of youth analyzed over time, preliminary findings are 
encouraging. Despite some expected obstacles related to teacher buy-in, communication, 
and other implementation issues, SPIRIT was successfully delivered to over 4,000 youth 
across the state of Missouri. Evaluation findings show slight improvements in reactive 
aggression, grades, disciplinary incidents, anti-social behavior, and substance use 
reduction. Future reports will explore differences between the five evidence-based 
programs in further detail and explore the programs’ abilities to create lasting effects for 
the Missouri youth being served by SPIRIT. 
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